Adaptive security camera image compression method of operation

ABSTRACT

A lossy compression method optimizes bandwidth and storage for a security surveillance network. An appliance on a local network attached to event capture terminals transforms image files into a key frame and at least one subsequent frame. Decompression combines a subsequent frame with its key frame to provide an image with graduated resolution/noise clutter. A camera records, and forwards a plurality of image files compatible with JPEG encoding. Key frames are selected from among the plurality of image files. A configurable low pass filter is reset for each train of a key frame and its subsequent frame or frames. Each low pass filter is selectively applied to each pixel block within a subsequent frame. The transformation operates on coefficients of frequency bins. Meta data enables decompression of a single subsequent frame by reversing some of the transformations to provide a JPEG compatible file having selectively reduced resolution or noise clutter.

CROSS-REFERENCES TO RELATED APPLICATIONS

This non-provisional continuation in part application claims the benefitof provisional application 61/949,526 and its priority date Mar. 7, 2014and parent application Ser. No. 14/479,349 Filed: Sep. 7, 2014.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

THE NAMES OF THE PARTIES TO A JOINT RESEARCH AGREEMENT

Not Applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISK OR ASA TEXT FILE VIA THE OFFICE ELECTRONIC FILING SYSTEM (EFS-WEB)

Not Applicable

STATEMENT REGARDING PRIOR DISCLOSURES BY THE INVENTOR OR A JOINTINVENTOR

Not Applicable

BACKGROUND OF THE INVENTION Technical Field

As is known, digital cameras record, store, and transmit images inseveral industry standard formats, such as H.234 and JPEG. The field ofthe invention is in the field of image analysis, particularly imagecompression or coding applied to products of Huffman and DCT encoding.

JPEG

As is known, the name “JPEG” stands for joint Photographic ExpertsGroup, the name of the committee that created the JPEG standard and alsoother still picture coding standards.

The JPEG standard specifies the codec (code decode transform), whichdefines how an image is compressed into a stream of bytes anddecompressed back into an image, but not the file format used to containthat stream.

JPEG uses a lossy form of compression based on the discrete cosinetransform (DCT). This mathematical operation converts each frame/fieldof the video source from the spatial (2D) domain into the frequencydomain (aka transform domain.) A perceptual model based loosely on thehuman psychovisual system discards high-frequency information, i.e.sharp transitions in intensity, and color hue. In the transform domain,the process of reducing information is called quantization. In simplerterms, quantization is a method for optimally reducing a large numberscale (with different occurrences of each number) into a smaller one,and the transform-domain is a convenient representation of the imagebecause the high-frequency coefficients, which contribute less to theover picture than other coefficients, are characteristicallysmall-values with high compressibility. The quantized coefficients arethen sequenced and losslessly packed into the output bitstream. Nearlyall software implementations of JPEG permit user control over thecompression-ratio (as well as other optional parameters), allowing theuser to trade off picture-quality for smaller file size.

JPEG File Interchange Format (JFIF) is a minimal file format whichenables JPEG bitstreams to be exchanged between a wide variety ofplatforms and applications. This minimal format does not include any ofthe advanced features found in the TIFF JPEG specification or anyapplication specific file format. Nor should it, for the only purpose ofthis simplified format is to allow the exchange of JPEG compressedimages.

JPEG Codec Example

Although a JPEG file can be encoded in various ways, most commonly it isdone with JFIF encoding. The encoding process consists of several steps:

1. The representation of the colors in the image is converted from RGBto Y′CBCR, consisting of one luma component (Y′), representingbrightness, and two chroma components, (CB and CR), representing color.This step is sometimes skipped. 2. The resolution of the chroma data isreduced, usually by a factor of 2. This reflects the fact that the eyeis less sensitive to fine color details than to fine brightness details.3. The image is split into blocks of 8.times.8 pixels, and for eachblock, each of the Y, CB, and CR data undergoes the Discrete CosineTransform (DCT). A DCT is similar to a Fourier transform in the sensethat it produces a kind of spatial frequency spectrum. 4. The amplitudesof the frequency components are quantized. Human vision is much moresensitive to small variations in color or brightness over large areasthan to the strength of high-frequency brightness variations. Therefore,the magnitudes of the high-frequency components are stored with a loweraccuracy than the low-frequency components. 5. The resulting data forall 8.times.8 blocks is further compressed with a lossless algorithm, avariant of Huffman encoding. 6. The decoding process reverses thesesteps, except the quantization because it is irreversible.DCT

As is known, the discrete cosine transform (DCT) expresses a finitesequence of data points in terms of a sum of cosine functionsoscillating at different frequencies. DCTs are important to numerousapplications in science and engineering, from lossy compression of audio(e.g. MP3) and images (e.g. JPEG) (where small high-frequency componentscan be discarded), to spectral methods for the numerical solution ofpartial differential equations. The use of cosine rather than sinefunctions is critical in these applications: for compression, it turnsout that cosine functions are much more efficient (fewer functions areneeded to approximate a typical signal), whereas for differentialequations the cosines express a particular choice of boundaryconditions.

In particular, a DCT is a Fourier-related transform similar to thediscrete Fourier transform (DFT), but using only real numbers. DCTs areequivalent to DFTs of roughly twice the length, operating on real datawith even symmetry (since the Fourier transform of a real and evenfunction is real and even), where in some variants the input and/oroutput data are shifted by half a sample. There are eight standard DCTvariants, of which four are common.

The most common variant of discrete cosine transform is the type-II DCT,which is often called simply “the DCT”, its inverse, the type-III DCT,is correspondingly often called simply “the inverse DCT” or “the IDCT”.

The compression method is usually lossy, meaning that some originalimage information is lost and cannot be restored, possibly affectingimage quality.

Like any Fourier-related transform, discrete cosine transforms (DCTs)express a function or a signal in terms of a sum of sinusoids withdifferent frequencies and amplitudes. Like the discrete Fouriertransform (DFT), a DCT operates on a function at a finite number ofdiscrete data points. The obvious distinction between a DCT and a DFT isthat the former uses only cosine functions, while the latter uses bothcosines and sines (in the form of complex exponentials).

Huffman Encoding/Decoding

As is known, Huffman coding is an entropy encoding algorithm used forlossless data compression. The term refers to the use of avariable-length code table for encoding a source symbol (such as acharacter in a file) where the variable-length code table has beenderived in a particular way based on the estimated probability ofoccurrence for each possible value of the source symbol.

Huffman coding uses a specific method for choosing the representationfor each symbol, resulting in a prefix code that expresses the mostcommon source symbols using shorter strings of bits than are used forless common source symbols. Huffman was able to design the mostefficient compression method of this type: no other mapping ofindividual source symbols to unique strings of bits will produce asmaller average output size when the actual symbol frequencies agreewith those used to create the code.

For a set of symbols with a uniform probability distribution and anumber of members which is a power of two, Huffman coding is equivalentto simple binary block encoding, e.g., ASCII coding. Huffman coding issuch a widespread method for creating prefix codes that the term“Huffman code” is widely used as a synonym for “prefix code” even whensuch a code is not produced by Huffman's algorithm.

Security Cameras

As is known, cameras used for security surveillance (SECCAMs) canproduce high resolution and low resolution digital files. The files maybe of static images or moving images. The cameras may auto focus andauto adjust for lighting. The cameras may store data or push it tonetwork attached servers. Many SECCAMs do not pan or zoom and alwayspresent the same view. Some SECCAMs take a still image when triggered bya sensor, a timer, or an event.

One problem with conventional SECCAMs installed with onsite storage isthat the data may be lost or stolen in the same incident for which theimages would have been useful. SECCAMs may provide timestamps at thecamera to record events for later viewing. In many cases, there is noattendant to monitor SECCAMs in real time. Thus economical remotestorage and delayed analysis are advantageous if the useful images couldbe transported without congesting a low bandwidth network. A requirementis that allocated bandwidth schedules not be exceeded. Within thispatent application we refer to JPEG compatible image files originatingor derived from SECCAMs as “previews” or “frames”. Applicantauthoritatively defines SubFrame, Jframe, and Kframe.

Thus it can be appreciated that what is needed is an improved system tobridge from cameras used for security surveillance to a networkconnected storage and analysis service center. Temporary storage ofcompressed images would allow bandwidth allocations to vary over timeand day.

BRIEF DESCRIPTION OF DRAWINGS

To further clarify the above and other advantages and features of thepresent invention, a more particular description of the invention willbe rendered by reference to specific embodiments thereof which areillustrated in the appended drawings. It is appreciated that thesedrawings depict only typical embodiments of the invention and aretherefore not to be considered limiting of its scope. The invention willbe described and explained with additional specificity and detailthrough the use of the accompanying drawings in which:

FIG. 1 is a block diagram of a conventional processor used forperforming method steps in an apparatus;

FIGS. 2-4 are flowcharts of steps in a method performed by a processor;

FIGS. 5-7 are block diagrams of an apparatus in a system; and

FIGS. 8-11 are method flowcharts of exemplary method embodimentsperformed by a processor.

SUMMARY OF THE INVENTION

A lossy compression method optimizes bandwidth and storage for asecurity surveillance network of appliances and event capture terminals.The method is performed by a processor upon an image file (Jframe)provided by a JPEG compatible encoder. The processor is adapted bycomputer readable instructions stored in non-transitory media to performoperations on Jframes in the spatial domain, the time domain and in thefrequency domain simultaneously which cannot be done mentally or byhand. Triggers cause the method to selectively reduce resolution oreliminate noise or overwrite selected frequency coefficients.

A method transforms a sequential plurality of Jframes into a traincomposed of a key frame (Kframe) and one or more subsequent frames(SubFrames). Each block of pixels in a key frame has coefficients in aplurality of frequency bins. Each corresponding block of pixels in asubsequent frame may contain the difference in coefficients between itssource Jframe and its Kframe. One or more subsequent frames follow a keyframe and are compressed relative to their untransformed Jframe bycontaining delta coefficients in the frequency domain. The resulting setof delta coefficients are re-encoded using a Huffman encoder.

Decompression combines a subsequent frame with its key frame to providean image with graduated resolution/noise clutter. While viewing asubsequent frame on its own is possible with a JPEG compatible browseror display it would seem to be meaningless without its reference keyframe. Reversing the transformation requires recovering all thefrequency coefficients of the key frame and the SubFrame and adding eachdelta coefficient to its corresponding Kframe coefficient. But theinvention further optimizes compression by removing thermal and visualnoise and sacrificing resolution in some of the pixel blocks. The methodfurther includes how to select pixel blocks and how to remove noise orsacrifice resolution.

A camera records, encodes, stores, and forwards a plurality of imagefiles, each file compatible with JPEG standard encoders/decoders(codecs). In addition to motion video or high definition or 3D or analogsignals, conventional cameras default to always providing a series ofstill images in digital format encoded by the JPEG standard. These arefrequency domain coefficients for blocks of pixels further encoded by aHuffman codec. A local area network such as Ethernet or Wi-Fi couplesthe camera with an appliance of the present disclosure.

Periodically or upon conditions in the content or external events, keyframes are selected from among the plurality of image files. Theapparatus may trigger on external events (e.g. sensors), image contents(e.g. motion, facial recognition), or internal processes such asperiodicity (e.g. timer, counter), or data stored (e.g. buffer capacity)to select a Jframe as a key frame. The characteristics of one key framemay influence the selection of a next key frame. The size of a key framemay influence the target size of its subsequent frames and theconfiguration of a low pass filter.

A configurable low pass filter is reset for each train of key frame andsubsequent frames. The method performed by the appliance uses the firstsubsequent frame following a key frame to configure a low pass filter. Atarget compressed file size is determined from the size of the keyframe. An integer value is determined for the extent of a mask whichremoves coefficients of high frequency bins. Clipping selectedfrequencies reduces noise in the transformed (and shrunken) SubFrame byreplacing certain frequency coefficients with zeros.

The configurable low pass filter (CLPF) is selectively applied to eachpixel block within a subsequent frame. The masking of coefficients mayoccur for none of, some of, or all of the pixel blocks of a subsequentframe. Separate rules may override the masking of coefficients in eachpixel block. As a result, a subsequent frame may contain pixel blockswhich have various resolution levels.

Meta data is stored separately or within the file headers of key framesand subsequent frames to enable decompression of a single subsequentframe. Meta data includes but is not limited to identification of keyframes and their locations (relative or absolute), the key frame eachsubsequent frame references, the size of files and the target size ortarget compression ratio, time and location, camera identity, and thelow pass filter configuration.

Typically circuits of a first appliance locally attached to a network ofevent capture terminals transforms and stores a plurality of imageframes into a key frame and at least one subsequent frame. The storageof transformed frames may be in locally attached storage or in a remotelocation referred to as the cloud. The result of each transformation isa JPEG compatible file but which may depend on decoding of a key frameto have visible content which is intelligible to a human viewer. If aJframe following a first key frame fails to meet its requiredcompression target, it is selected as a second key frame.

The transformation operates on coefficients of frequency bins. Oneaspect of the invention is to mask high frequency bins to achieve targetsize or compression ratios. Another aspect of the invention is to maskhigh frequency bins to remove thermal noise, lighting figments, andbackground vibration. Another aspect of the invention is to selectivelyimprove resolution of selected pixel blocks for pertinent securitysurveillance objectives.

Circuits of a second appliance coupled to a remote or local store oftransformed frames reverse some of the transformations to provide a JPEGcompatible file having selectively reduced resolution or noise clutter.The compressed stream of Kframes and SubFrames may be stored locallyand/or transmitted over a wide area network to one or more remote datacenters for access by conventional cloud services. Decompression of aSubFrame utilizes meta data to select its Kframe to reverse thetransformation of non-masked frequency coefficients and then Huffmanre-encode. The result may be viewed in any JPEG compatible browser ordisplay device. Advantageously, noise components of the original imagehave been selectively suppressed in storage, in transmission, and indisplay.

A first frame in a stream of JPEG image frames is designated a first keyframe and stored. Each frame subsequent to the first key frame istransformed into a delta frame containing the differences from the firstkey frame. A second frame is designated a second key frame when themagnitudes of the differences cause the effective compression of themost recent transformed subsequent frame to fall below a threshold.

Data compression is obtained by only storing into non-transitory mediaand transmitting over a network the differences between a non-baselineframe and its baseline predecessor. Taking advantage of the nature ofsecurity cameras, there is high potential for successful compression.Many images taken are the same as the previous image. Even if an imagehas something new, most of the image is background or foreground, whichhas not changed. Security cameras are generally fixed and do not pan,zoom, or change in attitude. Lighting due to time of day will change andpersons or vehicles will enter, leave, or transit from one edge toanother.

A new baseline is selected when compression falls below a target.Typically, the first frame after a key frame has the least change andprovides the best possible compression. The size of the difference fileswill range between the size of the first frame and that of the keyframe. A parameter is set to determine a target minimum compression,which when exceeded causes a new key frame to be selected.

By applying Huffman decoding, we recover coefficients for a DCTtransform of the image. These correspond to amplitudes for eachfrequency bin in a matrix of pixels. The amplitudes of a key frame arestored and the amplitudes of each subsequent image are subtracted fromthe corresponding coefficient. A small difference indicates little or nochange.

A sequence of baseline frames and difference frames are transmitted froma local network and stored in compressed form in a wide area networkattached cloud storage system which protects the security system fromloss of data by media destruction or theft.

New key frames are selected either from a timer or counter or when thefollowing compression steps do not provide substantial savings. Thefirst compression is to compare a frame to its key frame and determineonly the differences. If the differences indicate little change, eventhe resolution can be reduced by zeroing high frequency data for furthercompression. Even significant differences in high frequency data may bezeroed if analysis of low frequency or DC data indicates the source isstatic noise or vibration. Advantageously, an image may be a mix ofhigher and lower resolution blocks if analysis of low frequency or DCdata indicates that a substantial change is occurring in part of theimage but not another part of the image.

DETAILED DISCLOSURE OF EMBODIMENTS OF THE INVENTION

All of the following transformations and logic operations are performedby an electronic circuit. As is known, image file data may be read andstored to non-transitory computer readable media and operated on by aprocessor controlled by instructions in a store. The invention improvesthe performance and efficiency of a network and storage apparatuses usedin video surveillance by processes performed by executing instructionsstored in non-transitory media on data stored in non-transitory media.

A lossy compression method optimizes bandwidth and storage for asecurity surveillance network of appliances and event capture terminals.Taking advantage of the nature of security cameras, there is highpotential for successful compression. Many images taken are the same asthe previous image. Even if an image has some new content, most of theimage is background or foreground, which has not changed. Securitycameras are generally fixed and do not pan, zoom, or change in attitude.Lighting due to time of day will change gradually and persons orvehicles will enter, progress from one edge to another, and exit thefield of view.

An event capture terminal (such as a camera) records, encodes, stores,and forwards a plurality of image files, each file compatible with JPEGstandard encoders/decoders.

A plurality of image frames are transformed into a train made of a keyframe and one or more subsequent frames. By applying Huffman decoding,we recover coefficients for a DCT transform of the image. Thesecorrespond to amplitudes for each frequency bin in a matrix of pixels.The amplitudes of a key frame are stored and the amplitudes of eachsubsequent image are subtracted from the corresponding coefficient of ablock in its key frame.

Periodically or upon conditions in the content or external events, keyframes are selected from among the plurality of image files. New keyframes may be selected on a number of triggers. One could be triggeredby an external sensor, another from a timer or counter or file contentsuch as when the following compression steps do not provide substantialsavings. The first compression is to compare a frame to its key frameand utilize the differences in frequency coefficients. A furthercompression is available when the differences indicate little change, byzeroing high frequency data. Further masking of significant differencesin high frequency data applies when analysis of low frequency or DC dataindicates the source is static noise or vibration. Advantageously forbandwidth and storage conservation, a resulting transformed image may bea mix of higher and lower resolution blocks if analysis of low frequencyor DC data indicates that a substantial change is localized to a portionof the full image.

A configurable low pass filter is reset for each train of key frame andsubsequent frames. Typically, the first frame after a key frame has theleast change and provides the best possible compression. The size of thedifference files will range between the size of the first frame and thatof the key frame. A parameter is set to determine a target minimumcompression, which when exceeded causes a new key frame to be selected.

The low pass filter is selectively applied to each pixel block within asubsequent frame. If there is substantial amplitude in DC or lowfrequency bins, it suggests movement into or out of the field of view.If there is no significant difference in the DC or low frequency bins,the coefficients in the high frequencies can be discarded without lossof information because they are likely to show only static, or noise, orvibration. Pixel blocks with DC and low frequency activity causeretention of the high frequency coefficients to provide a higherresolution image. An unchanging background can be lower resolutionwithout affecting the security value. So a transformed frame mayusefully have a mix of higher resolution pixel blocks and lowerresolution pixel blocks. As is known, DC originally refers to directcurrent and distinguishes a signal value from AC or alternating current.By extension, in the frequency domain it refers to the lowest range offrequencies in a spectrum.

One detailed aspect of the invention is a method for compression of aplurality of JPEG codec compatible files (Jframe) in time, spatial, andfrequency domains which has the steps: reversing Huffman encoding of afirst Jframe and storing every frequency coefficient of every block ofpixels into a computer-readable store as a first key frame (Kframe);reversing Huffman encoding of an other Jframe to obtain every frequencycoefficient of every block of pixels; transforming the other Jframe intoa subsequent frame (SubFrame) by determining delta frequencycoefficients for each block of pixels in the SubFrame; storing all ofthe delta frequency coefficients of each block, and Huffman encoding thestored delta frequency coefficients of each block; when the resultingfile size meets the target compression goal, storing the file as asubsequent file and storing meta data relating the subsequent file toits related key file, and when the resulting file size exceeds thetarget compression goal, discarding the delta coefficients and storingthe original frequency coefficients of all the blocks as a new keyframe.

A delta coefficient is the difference between a frequency coefficient ofa block of pixels of a Jframe and the corresponding frequencycoefficient in its preceding Kframe.

This method performs compression in time by comparing a frame taken atone point in time with a reference frame from an earlier point in time.It also compresses spatially by operating on blocks of pixel in oneframe with the corresponding blocks of pixels in the reference frame.And the compression also operates in the frequency domain by subtractingeach DCT frequency coefficient from its equivalent.

Another process embodiment is a method for compression of a plurality ofJPEG codec compatible files (Jframe) in time, spatial, and frequencydomains which includes additional steps: reversing Huffman encoding of afirst Jframe and storing every frequency coefficient of every block ofpixels into a computer-readable store as a first key frame (Kframe);reversing Huffman encoding of an other Jframe to obtain every frequencycoefficient of every block of pixels; transforming the other Jframe intoa subsequent frame (SubFrame) by determining delta frequencycoefficients for each block of pixels in the SubFrame; applying a motiontrigger rule to the delta frequency coefficients of each block of pixelsseparately; on determining a condition that the motion trigger rulepasses for each block of pixels, storing all of the delta frequencycoefficients of each block, and Huffman encoding the stored deltafrequency coefficients of each block; on determining a condition thatthe motion trigger rule fails for each block of pixels, storing zerosfor all the delta frequency coefficients of each block; when theresulting file size meets the target compression goal, storing the fileas a subsequent file and storing meta data relating the subsequent fileto its related key file, and when the resulting file size exceeds thetarget compression goal, discarding the delta coefficients and storingthe original frequency coefficients of all the blocks as a new keyframe.

A motion trigger rule may involve the DC or the lowest frequency rangeof bins for a block of pixels. Above a threshold, a change from thereference coefficients of the key frame may activate a motion triggerrule.

In an embodiment, the method performed by a processor coupled tonon-transitory computer readable storage further includes: applying anoise trigger rule to the delta frequency coefficients of each block ofpixel separately; on determining a condition that the noise trigger rulefails for each block of pixels, storing all of the delta frequencycoefficients of each block, and Huffman encoding the stored deltafrequency coefficients of each block; on determining a condition thatthe noise trigger rule passes for each block of pixels, applying a HighFrequency Coefficient (HFC) mask which zeros the H highest frequencydelta coefficients of each block, and Huffman encoding the HFC maskeddelta frequency coefficients; and, determining a file size of a filewhich contains all the Huffman encoded delta frequency coefficientswhich have been HFC masked or not masked.

A noise trigger rule may compare high frequency components to lowfrequency components within a block of pixels. When delta low frequencycomponents are substantially the same or below a threshold, while deltahigh frequency components are above a threshold, it may activate a noisetrigger rule.

In another embodiment, the method further includes: applying a leastsignificant bit (LSB) trigger rule to the delta frequency coefficientsof each block of pixel separately; on determining a condition that theLSB trigger rule fails for each block of pixels, storing all of thedelta frequency coefficients of each block, and Huffman encoding thestored delta frequency coefficients of each block; on determining acondition that the LSB trigger rule passes for each block of pixels,applying an LSB mask which zeros the L least significant bits of thedelta coefficients of each block, and Huffman encoding the LSB maskeddelta frequency coefficients; and, determining a file size of a filewhich contains all the Huffman encoded delta frequency coefficientswhich have been LSB masked or not masked.

Within each frequency bin the least significant bits may not beperceptible to a human observer. A LSB trigger rule may be tailored foran application or a venue or time of day or the type of display oranalysis. A facial recognition backend may require more bits than ahuman operator can use. The rule may set L the number of bits maskedoff.

In an embodiment, the method also includes: configuring a low passfilter; applying a configured low pass filter trigger (CLPF) rule to thedelta frequency coefficients of each block of pixels separately; ondetermining a condition that the CLPF trigger rule fails for each blockof pixels, storing all of the delta frequency coefficients of eachblock, and Huffman encoding the stored delta frequency coefficients ofeach block; on determining a condition that the CLPF trigger rule passesfor each block of pixels, applying the configured low pass filter to thedelta frequency coefficients of each block, and Huffman encoding thefiltered delta frequency coefficients; and, determining a file size of afile which contains all the Huffman encoded delta frequency coefficientswhich have been filtered or not filtered.

A configured low pass filter rule may be triggered by motion, or noise,or contents of the file, or by external events such as sensors, e.g.sound, vibration, temperature. Once configured, the CLPF may be appliedon a block by block basis according to its trigger rule.

In an embodiment, configuring a low pass filter includes the steps:reading a target compression ratio from a non-transitory computerreadable store; determining a target compressed file size by applyingthe target compression ratio to a first key frame; determining a numberP of highest frequency bins of delta frequency coefficients which whenzeroed, cause a first subsequent frame to meet the target compressedfile size; and storing P as the configuration of a low pass filter untila new key frame is selected.

In certain circumstances more than one trigger rule may be activated andmore than one compression method may be applied to a frame or even to ablock of pixels within a frame.

Another aspect of the invention is in the embodiment of circuits of afirst appliance locally attached to a network of event capture terminalstransforms and stores a plurality of image frames into a key frame andat least one subsequent frame. Meta data is stored separately or withinthe file headers of key frames and subsequent frames to enabledecompression of a single subsequent frame.

A reception archival presentation method includes, receiving and storinga key frame; receiving and storing at least one difference frame;receiving a user command to display said difference frame; reading thekey frame from storage; decoding the coefficients of the key frame byHuffman decoding; reading the difference frame; decoding thecoefficients of the difference frame by Huffman decoding; adding thecoefficients of the key frame to the coefficients of the differenceframe; restoring a JPEG compatible file by Huffman encoding the sums ofthe coefficients; and transmitting the restored JPEG compatible file toa user workstation, whereby the user may display the restored JPEGcompatible file in a browser.

Referring now to the figures, a method embodiment is illustrated in FIG.2 to assist in apprehension of the subject matter. A method embodimentwhich is performed by a processor controlled by computer executableinstructions stored in non-transitory media includes: receiving a KeyFrame (Kframe), storing size, determining target size for subsequentframes, and recovering baseline frequency bin coefficients for everypixel block by reversing Huffman encoding of a JPEG codec compatibleimage file (Jframe) 201.

Several methods are disclosed infra to test a Jframe for selection as aKey Frame. Once selected, its frequency coefficients are used todetermine delta coefficients for subsequent frames.

The method continuing by receiving 1st Jframe following Kframe 310;while unlikely due to the nature of security surveillance systems, themethod includes determining not another Kframe 312; determining deltacoefficients for every frequency bin of every block of Jframe byreversing Huffman encoding and subtracting each frequency bincoefficient from its Kframe correspondent; 314 configuring Low PassFilter Mask to achieve target size for 1st subsequent frame 315 bysetting an integer to denote the number of high frequency bins that aremasked out up to a fixed minimum Low Pass Filter bandwidth (minLPFlimit); Huffman encoding delta coefficients to generate 1st SubsequentFrame (SubFrame); 319.

For each additional Jframe after the 1st SubFrame until a next Kframe isselected 400 the method continues: reversing Huffman encoding to recoverfrequency bin coefficients for every block; 420 determining deltacoefficients for every frequency bin of every pixel block of frame 430by reversing Huffman encoding and subtracting each frequency bincoefficient from its correspondent in the Kframe.

For each pixel block of the Jframe 500, one of the following steps:

Case I: When delta DC coefficient is greater or equal to a DC threshold,enabling all delta coefficients in the block; 540.

Case II: When delta DC coefficient is less than the DC threshold, and asum of a group of high AC delta coefficients exceeds an AC threshold,enabling only LPF unmasked delta coefficients in the block; 550.

Case III: When delta DC coefficient is less than the DC threshold, and asum of a group of high AC delta coefficients is less than or equal tothe AC threshold, forcing all delta coefficients in the block to zero;560 when all pixel blocks of the Jframe have been transformed, Huffmanencoding all the blocks to generate a next Subsequent Frame; 670 whichreturns the process to step 400; applying qualification test forselection as a Key Frame 780 which would return the process to step 201;and storing meta data relating each Subsequent Frame (SubFrames) to aKey Frame in the file headers or in a separate table 890.

It can be appreciated that the sequential exposition of the method stepsis presented for clarity and not as a limitation precluding embodimentprocesses that may be efficiently performed in parallel or differentorder by circuits or processor threads.

Various methods can be employed to select a Key Frame from among aplurality of JPEG codec compatible image files. In an embodiment, amaximum period in time or in number of frames can be set as a parameter.When that maximum is met, an image is selected as a new key frame. Allthe frames following will be compared to the key frame until a new keyframe is selected.

Exemplary Embodiments of the Claimed Subject Matter

One aspect of the invention is an exemplary apparatus having a circuitto receive JPEG codec compatible files and reverse Huffman encoding toobtain frequency coefficients for each pixel block; a circuit todistinguish key frames from subsequent frames; a circuit to set aconfigurable low pass filter for each train of a key frame and itssubsequent frame or frames; a circuit to determine delta coefficientsfor each frequency bin of each pixel block; a circuit to selectivelyzero insignificant delta coefficients; and a circuit to apply theconfigurable low pass filter to selected pixel blocks.

In an embodiment, the apparatus also has a circuit to Huffman encodedelta coefficients into a JPEG codec compatible file and store meta datato relate a subsequent frame to its key frame. In an embodiment, thecircuit to apply the configurable low pass filter is controlled by themagnitude of the delta coefficients in certain low frequency bins. In anembodiment, the circuit to set a configurable low pass filter determinesthe minimum number of high frequency coefficients to mask out to achievea target compression size.

Another aspect of the invention is an exemplary method for transformingJPEG codec compatible files by a processor communicatively coupled to atleast one event capture terminal and coupled to a non-transitorycomputer-executable instruction store, the method comprising: reading afirst target compression ratio store; receiving a first JPEG codeccompatible image file; determining a post-compression target file size;receiving a second JPEG codec compatible image file (Jframe); decodingfrequency bin coefficients for each pixel block of the first Jframe andof the second Jframe by reverse Huffman encoding; determining an integervalue to configure a low pass filter mask; determining a differencebetween each pair of respective coefficients of a pixel block in thefirst Jframe and its corresponding pixel block in the second Jframe;determining when the difference between each pair of respectivecoefficients of the low pass filter masked frequency bins is less than athreshold; and on the condition that the differences for all masked bitsare below the threshold; forcing the difference values of maskedfrequency bins to zero.

In an embodiment the process of determining an integer value toconfigure a low pass filter mask comprises: reading an integer value fora fixed minimum Low Pass Filter (minLPF) mask representing a limit to anumber of high frequency bins which may be set to zero, reading thepost-compression target file size; determining the smallest integervalue integer which successfully achieves the post-compression targetfile size; and storing the lesser of minLPF or the smallest integervalue which successfully achieves the post-compression target file sizeas the integer value to configure a low pass filter mask.

In an embodiment the method also includes determining that a 1st JPEGcodec compatible image file (Jframe) is a key frame by one ofperiodically or upon conditions in the content or external events,selecting a Jframe from among the plurality of image files by anexternal event, image content, periodicity in time or count, or quantityof data stored.

In an embodiment the method also includes storing metadata to identifythe relationship and location of key frames and subsequent frames.

In an embodiment the method also includes writing meta data into fileheaders for Kframes and SubFrames to identify their relationship.

In an embodiment the method also includes Huffman encoding thedifference values of all pixel blocks of the second Jframe to provide aJPEG compatible subsequent frame file.

Another aspect of the invention is an exemplary method for transformingJPEG codec compatible files by a processor communicatively coupled to atleast one event capture terminal and coupled to a non-transitorycomputer-executable instruction store, the method includes the steps:reading a first target compression ratio store; receiving a first JPEGcodec compatible image file; determining a post-compression target filesize; receiving a second JPEG codec compatible image file (Jframe);decoding frequency bin coefficients for each pixel block of the firstJframe and of the second Jframe by reverse Huffman encoding; selectingthe first Jframe as a key frame (Kframe) and the second Jframe as asubsequent frame (SubFrame); differencing coefficients of a key frameand a subsequent frame; re-encoding the difference in coefficients as aJPEG compatible subsequent frame; determining when the resultantsubsequent frame meets the required target size; and on the conditionthat the resultant subsequent frame meets the required target size,storing the file and annotating the meta data with the relationship ofthe subsequent file to the key file; or on the condition that theresultant subsequent frame fails to meet the required target size,discarding it and designating the second Jframe as a second key frame.

In an embodiment, the method also includes determining the absolutemagnitude of the difference between the coefficients of certain lowfrequency bins of the key frame and the subsequent frame; summing theabsolute magnitude of the differences between the coefficients ofcertain low frequency bins; on the condition when the sum of theabsolute magnitude of the coefficients of certain low frequency binsexceeds a threshold, enabling all of the differences betweencoefficients in all of the frequency bins to be available forre-encoding into a JPEG compatible file.

In an embodiment the method also includes determining the absolutemagnitude of the difference between the coefficients of certain lowfrequency bins of the key frame and the subsequent frame; summing theabsolute magnitude of the differences between the coefficients ofcertain low frequency bins; on the condition when the sum of theabsolute magnitude of the coefficients of certain low frequency bins isbelow a threshold, enabling only the coefficients of the low pass filtermask frequency bins for re-encoding into a JPEG compatible file.

In an embodiment the method also includes reading an integer value for afixed minimum Low Pass Filter (minLPF) mask representing a limit to anumber of high frequency bins which may be set to zero; reading thepost-compression target file size; determining the smallest integervalue integer which successfully achieves the post-compression targetfile size; storing the lesser of minLPF or the smallest integer valuewhich successfully achieves the post-compression target file size as theinteger value to configure a low pass filter mask; and designating thenon-masked bits of the low pass filter as indicators of certain lowfrequency bins.

In a useful application which illustrates the benefits and theprinciples of the subject of the application a local area networkcouples security cameras to a compression apparatus. It is desirablethat the long-term image storage be offsite to protect against theft ordestruction. Lower resolution image files in JPEG compatible fileformats are pushed across a public wide area network to a cloud storageservice. To avoid congestion, a maximum bit rate is allocated to thesecurity service at all times. The apparatus adaptively applies a set ofcompression rules and methods to meet the allocated budget for imagetransmission.

Another method of file compression includes masking a number of leastsignificant bits of delta frequency coefficients to an efficientlyencoded pattern, such as but not limited to all ones or all zeros.

In a first embodiment, the file sizes of a key frame and a subsequentframe are compared to a desired compression ratio. When the desiredcompression ratio is not met, a number C is iterated to mask C leastsignificant bits up to attaining an efficiently encoded pattern. When Cresults in a desired compression ratio it is stored for use until thenext key frame is selected.

In a second embodiment, a post compression application determines alevel of precision necessary for operation. Facial recognition,character recognition, vehicle identification, handwriting recognitionare examples of applications that require a level of precision. A numberA is selected to optimize performance of an application by masking Aleast significant bits up to attaining a desired success rate for theApplication.

In a third embodiment, an image noise trigger determines a level ofprecision which is significant, beyond which variation is random andmeaningless. A number N is selected to mask least significant bits ofselected frequency coefficients which correspond to random andmeaningless variations. Time of day, lighting, weather and electronicdevice sensitivity are inputs to selection of the value of N.

In a fourth embodiment, a motion trigger operates to select pixel blocksin which M least significant bits of high frequency coefficients aremasked. Pixel blocks which do not trigger a motion detector may befurther compressed than pixel blocks which evidence motion by DC andlower frequency coefficients.

An apparatus receives image frames from security video cameras. If theimages are already encoded using a JPEG compatible codec, they aretransformed by Huffman decoding to produce a plurality of blocks havingamplitudes as frequency coefficients.

Compression is controlled by a set of parameters which includesselection of key frames. A change exceeding a threshold inlower-frequency amplitudes, determined by a set of parameters, controlsnoise filtering. Changes in higher frequency coefficients are notencoded when the size of images exceeds a target.

After high-frequency amplitudes have been clipped to remove noise fromselected blocks, the remaining frequency amplitudes of the subsequentframe is compared with a key frame and a delta frame is determined. Theamount of compression achieved by the delta is computed and tracked.Each delta is transformed by Huffman encoding and stored as a JPEGunless the amount of compression is decreasing. On that condition, a newkey frame is selected.

One aspect of the present invention performs, by a processor, deltacompression of jpeg preview images. A key frame is stored innon-transitory media, and a subsequent frame is encoded as deltas fromthat key frame. The amount of compression being achieved is tracked, andwhen it starts decreasing a new key frame is generated. The delta framesare also encoded as jpeg compatible files, so they also benefit from theJPEG codec's compression/encoding gains. Thus, if one displays a rawdelta frame, it appears as an all grey preview because each block offrequency coefficients are difference values between coefficients of akey frame and coefficients of a subsequent image frame. For staticscenes, the embodiment has achieved about 4.times. compression. All ofthis is “unwound” within the archiver, so has no impact on externalAPIs.

All the difference transformations are done in frequency space, whichadvantageously requires only Huffman encoding/decoding of JPEG fileswhich is very fast. In addition, when motion detection is performed inDCT space, decoding all previews to this level can be done only once.

JPEG files are encoded in a “spatial frequency” dimension. At night,cameras normally move to very high gain settings, which creates“speckle” noise—each pixel has small relatively random noise added toit. This creates a lot of high frequency noise and cause JPEG (and anyother video compression scheme) to produce much larger images. In thepresent invention, the method arbitrarily “clips” this data, notencoding changes in the higher frequencies when it determines the sizestarting to grow out of control.

While this clipping it normally invisible, where motion is actuallyoccurring the clipping can significantly degrade the utility of theresulting image. To avoid this, if a JPEG block has significant changein the lower frequency coefficients the clipping is prevented. Contraryto privacy considerations, it is the home intruder who is kept inhighest definition while the pixel blocks of static room occupants arecompressed into lower resolution.

A transmission method is disclosed: receiving a bandwidth and schedulebudget; receiving a plurality of JPEG compatible image files from asecurity surveillance camera; determining a key frame and a differenceframe; storing the key frame; compressing and storing said differencefiles in non-transitory media; determining the schedule enablestransmission; and transmitting the key frame and the difference frame toa storage server over a wide area network.

A currently preferred exemplary embodiment follows:

One aspect of the present invention performs, by a processor, deltacompression of JPEG preview images. A key frame is stored innon-transitory media, and a subsequent frame is encoded as deltas fromthat key frame. The amount of compression being achieved is tracked, andwhen it starts decreasing a new key frame is generated. The delta framesare also encoded as JPEG compatible files, so they also benefit from theJPEG codec's compression/encoding gains. Thus, if one displays a rawdelta frame, it appears as an all grey preview because each block offrequency coefficients are difference values between coefficients of akey frame and coefficients of a subsequent image frame. For staticscenes, the embodiment has achieved about 4.times. compression. All ofthis is “unwound” within the archiver, so has no impact on externalAPIs.

All the difference transformations are done in frequency space, whichmeans it only requires Huffman encoding/decoding of JPEG files which isvery fast. In addition, when motion detection is performed in DCT space,decoding all previews to this level can be done only once.

Another aspect of the invention is a method for supporting transfer ofsecurity camera images through a limited bandwidth channel to remotestorage services comprising: receiving from a camera an image fileencoded by a JPEG compatible codec; reversing Huffman encoding toreconstitute a plurality of frequency coefficients for 8.times.8 blocksof pixels; subtracting each coefficient from a corresponding coefficientstored for a predecessor key frame to determine difference results; whenthe difference results are small relative to a threshold, setting highfrequency coefficients to zero; Huffman encoding the differencecoefficients; and transmitting the encoded difference coefficients as adifference file.

In an embodiment, the method further comprises when the differenceresults are high relative to a threshold; determining the image file isa new key file; storing the coefficients; and transmitting the imagefile to remote storage services as a new key file.

In an embodiment, the method further comprises when the differenceresults for low frequency coefficients is small and the differenceresults for high frequency coefficients are high, determining thepattern corresponds to static noise or vibration and setting the highfrequency difference results to zero; when the difference results forlow frequency coefficients is large and the difference for highfrequency coefficients are high, determining the pattern corresponds tomovement and storing the high frequency differences as computed.

In an embodiment, the method further comprises reading fromnon-transitory media a minimum threshold of compression parameter;determining a ratio of compression for each Huffman encoded differencefile to its key file; and on the condition that the ratio passes theminimum threshold, transmitting the difference file to remote storage,or on the condition that the ratio fails the minimum threshold,transmitting the image file to remote storage and storing itscoefficients as a new key file.

One aspect of the invention is a method for supporting transfer ofsecurity camera images through a limited bandwidth channel to remotestorage services comprising: receiving from a camera an image fileencoded by a JPEG compatible codec; reversing Huffman encoding toreconstitute a plurality of frequency coefficients for 8.times.8 blocksof pixels; subtracting each coefficient from a corresponding coefficientstored for a predecessor key frame to determine difference results; whenthe difference results are small relative to a threshold, setting highfrequency coefficients to zero; Huffman encoding the differencecoefficients; and transmitting the encoded difference coefficients as adifference file.

In an embodiment, the method further comprises when the differenceresults are high relative to a threshold; determining the image file isa new key file; storing the coefficients; and transmitting the imagefile to remote storage services as a new key file.

In an embodiment, the method further comprises when the differenceresults for low frequency coefficients is small and the differenceresults for high frequency coefficients are high, determining thepattern corresponds to static noise or vibration and setting the highfrequency difference results to zero; when the difference results forlow frequency coefficients is large and the difference for highfrequency coefficients are high, determining the pattern corresponds tomovement and storing the high frequency differences as computed.

In an embodiment, the method further comprises reading fromnon-transitory media a minimum threshold of compression parameter;determining a ratio of compression for each Huffman encoded differencefile to its key file; and on the condition that the ratio passes theminimum threshold, transmitting the difference file to remote storage,or on the condition that the ratio fails the minimum threshold,transmitting the image file to remote storage and storing itscoefficients as a new key file.

Reference will now be made to the drawings to describe various aspectsof exemplary embodiments of the invention. It should be understood thatthe drawings are diagrammatic and schematic representations of suchexemplary embodiments and, accordingly, are not limiting of the scope ofthe present invention, nor are the drawings necessarily drawn to scale.

Referring now to FIG. 3, another embodiment of the present invention isa method performed by a computer processor when executing instructionsstored in non-transitory computer readable media, the method fordetermining a compressed Jframe 200. By Jframe we mean an image filerecorded by a security surveillance camera and stored as a JPEG codeccompatible file. The method includes reading a stored minimumcompression threshold parameter 301; receiving an image frame from asecurity camera 310; Huffman decoding each frequency coefficient 330 foreach block of pixels in the image; determining a difference from a keyframe for each frequency coefficient 340 for every block; determining aSubFrame by Huffman encoding the difference in frequency coefficientsand computing total size 350; when a frame counter identifies the KthSubFrame following the previous Key frame, determining compression 360;when the Kth SubFrame meets the minimum compression threshold, storingthe size as a target 370; and when the SubFrame exceeds the minimumcompression threshold, select a new key frame 390.

Referring now to FIG. 4, an aspect of the invention is a method by aprocessor applying a test for selection as Key Frame 780. The methodincludes decrementing a Jframe counter or timer from the last key framewhich counts down to zero 410; receiving an image Jframe from a securitycamera 420; Huffman decoding each frequency coefficient 313 of a blockin the image Jframe; determining a difference from previous key framefor each frequency coefficient 314; Huffman encoding the differences infrequency coefficients 319; determining a storage size for thecompressed Jframe 460; when the compressed storage size exceeds a targetsize, storing the received image Jframe as a new key frame 470; storingall the frequency coefficients of the new key frame and the size of thenew key frame 480; and resetting Jframe counters to K+1 and timers toT+1 490.

Referring now to FIG. 5, an apparatus embodiment 500 of the presentinvention includes an image frame receiver 701 which receives JPEGcompatible files from a plurality of network attached securitysurveillance cameras (not shown). The image frame receiver is coupled toa subsequent frame store 710. The subsequent frame store is coupled to aHuffman decoder 720 which reverses part of the JPEG encoding process toyield coefficients for each block of pixels. The coefficients whichcorrespond to amplitudes of a DCT transformation are stored into theattached subsequent coefficient store 730. A key frame has previouslybeen selected, decoded, and stored. Its respective coefficients are readfrom a key coefficient store 740. Both stores are coupled to acoefficient differencer circuit 750 which subtracts the coefficients ofthe subsequent frame from the coefficients of the key frame. Thecoefficient differencer 750 is coupled to a low pass filter circuit 760which zeros the amplitudes of selected high frequency terms. That is,the high frequency components of a subsequent frame are discarded. Theremaining coefficients are Huffman encoded to create a JPEG compatiblefile but which only has the differences of certain coefficients betweenthe key frame and the subsequent frame. A key switch trigger 975 maydetermine that the degree of compression is satisfactory. The Huffmanencoded difference file is stored into a transmit buffer 779 foreventual uploading through a wide area network to a storage service.

Referring now to FIG. 6, an apparatus embodiment 600 of the presentinvention also includes a DC level trigger circuit 855 which is coupledto the output of the coefficient differencer 750, and to the input ofthe low pass filter 760. In an embodiment the DC level trigger is alsocoupled to the Huffman encoder 770 (not shown). Depending on variousstored parameters, also not shown, the DC level trigger enables ordisables the low pass filter 760 to zero selected high frequencycomponents. That is, the DC level trigger enables noise filtering on thecondition that the DC or low frequency components of the image are notsubstantially different from the key frame. When they are substantiallydifferent, a higher resolution block of pixels is passed to the Huffmanencoder 770. Beneficially, those blocks of the image which aresubstantially unchanging will have noise reduction and those blocks withsignificant change will have higher quality of resolution. Illustratedin FIG. 6 is the scenario in which the increased data of some blocks hasnot grown to a level which causes the key switch trigger 975 to activateso the resulting mixed resolution image is stored into the transmitbuffer 779.

Referring now to FIG. 7, an apparatus embodiment 700 of the presentinvention also includes a key image switch circuit 776 which is coupledto the subsequent frame store 710 and a key frame store 916, and iscoupled to the subsequent coefficient store 730 and the key coefficientstore 740. The key image switch 776 is controlled by a key switchtrigger circuit 975 which is coupled to the Huffman encoder 770. The keyframe store 916 is further coupled to a transmit buffer 779. A new keyframe may be selected for a number of reasons: there may be a counter ortimer to establish a maximum period for transmitting a key frame fromwhich all subsequent frames are differenced. But a beneficial aspect ofthe present invention is to adaptively select key frames to performcompression on a series of JPEG compatible image frames. In the exampleshown, the DC level trigger 855 determines that one or more blocks ofthe image have substantially changed and disables the low pass filter760 from zeroing high frequency coefficients. After the encoding of thedifference frame in the Huffman encoder 770, the size of the result andthe degree of compression attained is compared with a target size by thekey switch trigger 975. When the key switch trigger determines that thecompression is no longer sufficiently advantageous (according to storedparameters not shown), the key image switch selects the currentsubsequent frame as a new key frame by transferring the subsequent framefrom the subsequent frame store 710 into the key frame store 916 andalso to the transmit buffer 779. The contents of the subsequentcoefficient store 730 are also transferred to the key coefficient store740. Embodiments of the circuits may also read out the subsequent framestore 710 directly to a transmit buffer 779 as the following subsequentframe is read in. Circuit functionalities herein disclosed may becombined into more complex circuits or parallel processor instructionswithout changing the claimed invention.

FIG. 8 is a flowchart of a method 800 embodiment of the inventionwherein a processor to transforms image files by adaptively compressinga series of security camera frames for transmission through networkapparatuses and storage into non-transitory media, at least performing:receiving a sequential plurality of JPEG encoded image files (Jframes)810; determining DCT coefficients for every Jframe 820; selecting atleast one key frame (Kframe) from among the Jframes 830; transformingunselected Jframes into subsequent frames (SubFrames) with respect to anantecedent Kframe 840; conditioning DCT coefficients for each SubFrame850; and storing Kframes and their SubFrames after conditioning 860. Forthis patent application we define conditioning to mean overwriting oneor more bits of one or more frequency bins. For example, when deltacoefficients are randomly near zero, there is less value to preservingthem especially when imperceptible to a user. A default compressionscheme can be overridden by other triggers which are content orextrinsically driven.

Exemplary triggers to selection of key frames include but are notlimited to: selecting each key frame from a sequential plurality ofJframes is triggered by a clock/calendar 832; selecting each key framefrom a sequential plurality of Jframes is triggered by a counter 834;selecting each key frame from a sequential plurality of Jframes istriggered by an external sensor 836; and selecting each key frame from asequential plurality of Jframes is triggered by reaching a threshold ofdeficient file compression of any subsequent frame relative to itsrespective key frame 838.

In an embodiment, the conditioned delta coefficients are transformedinto JPEG codec compatible files for transmission or storage. Whiletechnically compatible with browsers they would not be meaningfulwithout reference to an antecedent key frame. Therefore, storing Kframesand their SubFrames after conditioning includes Huffman encodingconditioned delta DCT coefficients for every pixel block of a SubFrameinto JPEG codec compatible files 861; storing meta data relating eachSubFrame to its respective Kframe 862; and storing the base line DCTcoefficients of each Kframe 863.

Referring now to FIG. 9 the invention enables mixed resolution andheterogeneous compression by considering the content of each block ofpixels. In an embodiment, determining DCT coefficients for every Jframe920 includes reverse Huffman encoding each block of pixels to determinecoefficients for each frequency bin 921; wherein transforming unselectedJframes into SubFrames comprises determining delta coefficients 940 bysubtracting every unselected Jframe coefficient from its respectiveKframe coefficient 941; and wherein conditioning DCT coefficients foreach SubFrame 950 comprises: configuring a low pass filter 960; andselecting pixel blocks in each SubFrame to condition 970.

A low pass filter blocks data associated with high frequencies orcoefficients of high frequency bins. A configurable low pass filteradjusts the number of frequency bins that pass through at the low end orare blocked at the high end.

Another embodiment includes configuring a low pass filter 960 byreceiving a target compression ratio 967; determining a number P ofhighest frequency bins which when zero'ed, cause selected pixel blocksto meet a target compression ratio 968; and storing P for use inconditioning DCT coefficients 969.

Several ways to determine which pixel blocks of an unselected Jframe areconditioned are disclosed, firstly: wherein selecting pixel blocks 970in each SubFrame to condition includes: determining a motion conditionin each block of pixels 971; selecting a block of pixels in which themotion condition is false 973; and enabling a configurable low passfilter for said selected blocks of pixels 976, whereby higher resolutionis reserved for pixel blocks that evidence motion relative to its keyframe.

In another embodiment, selecting pixel blocks 970 in each SubFrame tocondition includes: determining a motion condition in each block ofpixels 972; selecting a block of pixels in which the motion condition istrue 974; and disabling a configurable low pass filter for selectedblocks of pixels 976, whereby higher resolution is provided for pixelblocks that are changing.

Now referring to FIG. 10 which discloses more about a conditioning,method 1000 of claim 6 wherein conditioning DCT coefficients for eachSubFrame further includes: configuring a low pass filter with an integerF 1011; determining delta frequency coefficients for each subsequentframe 1013; receiving a minimum F integer for number of low frequencybins to be passed through at least 1015; receiving a desired compressionratio between a subsequent frame and its key frame 1017; and applyingthe configured low pass filter to delta coefficients of a pixel block1019.

In addition, an embodiment also includes determining a file size for asubsequent frame after Huffman encoding 1023; on the condition thedesired compression ratio is met, storing the subsequent frame with metadata pointing to its key frame 1025; on the condition the desiredcompression ratio is not met, selecting the Jframe as a next key frame1027.

In addition, an embodiment also includes: determining delta coefficientsfor F low frequency bins 1032; when delta coefficients for F lowfrequency bins are insignificant, incrementing F until non-trivial deltacoefficients are within scope 1036; and configuring the low pass filterto the post incremented F 1038, whereby the configurable filter passes aminimum dynamic range of non-zero significant bits.

In addition, an embodiment also includes resetting configurable LPF uponeach Kframe selection 1042, whereby the F integer of low frequency binsto be passed through is increased up to the limit of desired compressionratio 1046; and storing the increased F integer until a new Kframe isselected 1048.

In addition, an embodiment also includes selecting pixel blocks forconditioning 1054; applying the configurable low pass filter to selectedpixel blocks 1056; and storing conditioned DCT coefficients as a Jframeafter Huffman encoding 1058, wherein selecting pixel blocks is triggeredby motion, preselection of views, facial recognition, or externalsensors.

Referring now to FIG. 11, the invention also includes conditioning byLSB masking 1100 of subsequent frames, wherein LSB masking includes:reverse Huffman coding of a plurality of JPEG codec compatible imagefiles 1162; subtracting each DCT frequency coefficient from itscorresponding key frame equivalent to determine a delta frequencycoefficient 1164; and overwriting a number of least significant bits ofdelta frequency coefficients with an efficiently compressible bitpattern 1166, whereby file compression is achieved by eliminatingimperceptible and unnecessary precision. Repeating ones or zeros areoften efficiently compressed but possibly extending an establishedpattern is just as efficient.

In an embodiment, LSB masking is applied to C least significant bits ofdelta coefficients to meet a desired compression ratio between asubsequent frame and its key frame comprising: receiving a desiredcompression ratio 1171; determining delta frequency coefficients for afirst subsequent frame following a key frame 1173; iterating a number Cto mask C least significant bits of delta frequency coefficients up toattaining the desired compression ratio 1175; storing resultant number Cuntil a new key frame is selected 1177; and masking C least significantbits of each subsequent frame until a new key frame is selected 1179.Excess precision may be imperceptible to human eyes and can beeliminated to achieve desired compression. There is unlikely to be realchange between a key frame and its immediately following frame sotrimming off least significant bits of all delta frequencies to meet acompression goal has low risk of error or omission.

In an embodiment, LSB masking is applied to A least significant bits ofdelta coefficients to optimize performance of a post-compressionapplication wherein A is selected from among values appropriate toattain a desired success rate for said application which requires arange of precision 1182. Depending on the type of application it may beunnecessary to have really detailed images. People who are familiar toan observer can be identified in pretty low resolution photographs.Automated facial recognition requires specific measurements forapplication of a database. License plate or character recognition isfairly efficient. Each application can use experimental methods todetermine when an acceptable success rate will be achieved.

In an embodiment, LSB masking is applied to N least significant bits ofdelta coefficients of selected frequency bins on a condition of imagenoise triggered by rapid random and non-linear variations in highfrequency bins while DC and low frequency bins are stable wherein N isselected from among values appropriate for device sensitivity tolighting, time of day, and weather 1184. That is both motion and staticare measured in high frequency bins but motion is ruled out if lowfrequency bins are quiescent so the level of precision is reduced.

In an embodiment, LSB masking is applied to M least significant bits ofhigh frequency delta coefficients of a pixel block on a condition ofmotion detection triggered by DC and low frequency delta coefficients ofsaid pixel block 1186. When some blocks contain motion then some binswill have more sensitivity and there will be greater precision.

CONCLUSION

A system provides a cloud based archival and presentation apparatuscommunicatively coupled by a unreliable low bandwidth wide area networkto a transmission apparatus locally networked to a plurality ofconventional security video capture devices. The transmission apparatusreceives a sequence of images, determines motion, removes noise, selectskey frames, compresses interstitial frames, and transmits compressedjpeg previews to the archival and presentation apparatus.

One aspect of the invention is a method for operation of a systemperformed at processors executing computer-instructions encoded innon-transitory media on data stored in electronic circuits by: at atransmission apparatus coupled to a plurality of security video capturedevices, receiving image files split into blocks of 8.times.8 pixels,transformed by a Discrete Cosine Transform (DCT) into frequency bins. ADCT is similar to a Fourier transform in the sense that it produces akind of spatial frequency spectrum; decompressing the data bytransforming the image file components by applying a variant of Huffmanencoding; reading the amplitudes of the frequency components whichinclude high-frequency components, mid-frequency components, andlow-frequency components; determining motion; reading stored parameterswhich control noise suppression; suppressing noise components whendetermining motion is not true; reading stored parameters which controlselection of key frames; selecting a first key frame; comparing aplurality of subsequent frames to the first key frame; determining adelta frame between each subsequent frame and a first key frame;determining an amount of compression which each delta frame has over thefirst key frame, while the amount of compression is not decreasing foreach subsequent frame, encoding and storing the delta frame with JPEGtransformation; while the amount of compression is decreasing,discarding the delta frame and selecting a second key frame; andtransmitting key frames and delta frames to an archival apparatus.

An apparatus adaptively compresses image files to fulfill a storage orbandwidth budget. Portions of a JPEG codec compatible file arecompressed by suppressing noise or by selectively sacrificingresolution. An image frame is selected to serve as a baseline orreference. Compression may be obtained by operations in the frequencydomain by comparing the magnitude of frequency coefficients and byselective application of a low pass filter.

The present invention may be easily distinguished from conventionalvideo compression or file compression methods and apparatus byselectively applying a configurable low pass filter to pixel blocks.

The configurable low pass filter is adapted by operating on a key frameand its first subsequent frame to determine a mask for certain highfrequency bin coefficients toward achieving a target compression ratioor size.

The present invention can be easily distinguished from H.264 which isblock-oriented motion-compensation-based video compression standarddeveloped by the ITU-T Video Coding Experts Group (VCEG) together withthe ISO/IEC jTC1 Moving Picture Experts Group (MPEG). As is known, H.264encoding and decoding requires significant computing power in specifictypes of arithmetic operations. While security camera themselves maythemselves contain H.264 encoders, it is much more difficult to decode,adaptively compress, and reencode H.264 to meet a bit rate quota.Pushing H.264 files into a channel would disadvantageously swamp a widearea network.

The present invention can be easily distinguished by being dynamicallyself-adjusting to meet a substantially lower target bit rate. Thepresent invention can be easily distinguished from conventionalcompression by providing different resolution to portions of the sameimage.

The present invention can be easily distinguished from conventionalinterframe compression by its operation in the frequency domain. Deltasare determined from a key frame and a subsequent frames. While theamount of compression is the same or increasing, the delta are encodedand stored. When the amount of compression decreases below a threshold,a new key frame is selected. The present invention is distinguished byusing low-frequency coefficients of each block of pixels to determinewhen to control noise introduced by high gain settings. The presentinvention is distinguished by selecting key frames as a function ofdecreasing compression.

The techniques described herein can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or incombinations of them. The techniques can be implemented as a computerprogram product, i.e., a computer program tangibly embodied in aninformation carrier, e.g., in a machine-readable storage device or in apropagated signal, for execution by, or to control the operation of,data processing apparatus, e.g., a programmable processor, a computer,or multiple computers. A computer program can be written in any form ofprogramming language, including compiled or interpreted languages, andit can be deployed in any form, including as a stand-alone program or asa module, component, subroutine, or other unit suitable for use in acomputing environment. A computer program can be deployed to be executedon one computer or on multiple computers at one site or distributedacross multiple sites and interconnected by a communication network.

Method steps of the techniques described herein can be performed by oneor more programmable processors executing a computer program to performfunctions of the invention by operating on input data and generatingoutput. Method steps can also be performed by, and apparatus of theinvention can be implemented as, special purpose logic circuitry, e.g.,an FPGA (field programmable gate array) or an ASIC (application-specificintegrated circuit). Modules can refer to portions of the computerprogram and/or the processor/special circuitry that implements thatfunctionality.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto-optical disks, or optical disks. Information carrierssuitable for embodying computer program instructions and data includeall forms of non-volatile memory, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in special purposelogic circuitry.

FIG. 1 is a block diagram of an exemplary processor that may be used toperform one or more of the functions described herein. Referring to FIG.1, processor 100 may comprise an exemplary client or server process.Processor 100 comprises a communication mechanism or bus 111 forcommunicating information, and a processor core 112 coupled with bus 111for processing information. Processor core 112 comprises at least oneprocessor core, but is not limited to a processor core, such as forexample, ARM™, Pentium™, etc.

Processor 100 further comprises a random access memory (RAM), or otherdynamic storage device 104 (referred to as main memory) coupled to bus111 for storing information and instructions to be executed by processor112. Main memory 104 also may be used for storing temporary variables orother intermediate information during execution of instructions byprocessor core 112.

Processor 100 also comprises a read only memory (ROM) and/or otherstatic storage device 106 coupled to bus 111 for storing staticinformation and instructions for processor core 112, and anon-transitory data storage device 107, such as a magnetic storagedevice or flash memory and its associated control circuits. Data storagedevice 107 is coupled to bus 111 for storing information andinstructions.

Processor 100 may further be coupled to a display device 121 such a flatpanel display, coupled to bus 111 for displaying information to acomputer user. Voice recognition, optical sensor, motion sensor,microphone, keyboard, touch screen input, and pointing devices 123 maybe attached to bus 111 or a wireless network interface 125 forcommunicating selections and command and data input to processor core112.

A number of embodiments of the invention have been described.Nevertheless, it will be understood that various modifications may bemade without departing from the spirit and scope of the invention. Forexample, other network topologies may be used. Accordingly, otherembodiments are within the scope of the following claims.

The invention claimed is:
 1. A method performed by a processor totransform image files by adaptively compressing a series of securitycamera frames for transmission through network apparatuses and storageinto non-transitory media, comprising: receiving a sequential pluralityof Joint Photographic Experts Group standard (JPEG) encoded image files(Jframes); determining discrete cosine transform (DCT) coefficients forevery Jframe; selecting at least one key frame (Kframe) from among theJframes; transforming unselected Jframes into subsequent frames(SubFrames) with respect to an antecedent Kframe; conditioning DCTcoefficients for each SubFrame; and storing Kframes and their SubFramesafter conditioning, wherein selecting each key frame from a sequentialplurality of Jframes is triggered by reaching a threshold of deficientfile compression of any subsequent frame relative to its respective keyframe.
 2. A method performed by a processor to transform image files byadaptively compressing a series of security camera frames fortransmission through network apparatuses and storage into non-transitorymedia, comprising: receiving a sequential plurality of JointPhotographic Experts Group standard (JPEG) encoded image files(Jframes); determining discrete cosine transform (DCT) coefficients forevery Jframe; selecting at least one key frame (Kframe) from among theJframes; transforming unselected Jframes into subsequent frames(SubFrames) with respect to an antecedent Kframe; conditioning DCTcoefficients for each SubFrame; and storing Kframes and their SubFramesafter conditioning, wherein determining DCT coefficients for everyJframe comprises: reverse Huffman encoding each block of pixels todetermine coefficients for each frequency bin; wherein transformingunselected Jframes into SubFrames comprises: determining deltacoefficients by subtracting every SubFrame coefficient from itsrespective Kframe coefficient; and wherein conditioning DCT coefficientsfor each SubFrame comprises: selecting pixel blocks in each SubFrame tocondition.
 3. The method of claim 2 wherein selecting pixel blocks ineach SubFrame to condition comprises: determining a motion condition ineach block of pixels; selecting a block of pixels in which the motioncondition is false; and enabling a configurable low pass filter for saidselected blocks of pixels, whereby higher resolution is reserved forpixel blocks that evidence motion relative to its key frame.
 4. Themethod of claim 2 wherein selecting pixel blocks in each SubFrame tocondition comprises: determining a motion condition in each block ofpixels; selecting a block of pixels in which the motion condition istrue; and disabling a configurable low pass filter for selected blocksof pixels, whereby higher resolution is provided for pixel blocks thatare changing.
 5. The method of claim 2 further comprising: configuring alow pass filter by receiving a target compression ratio; determining anumber P of highest frequency bins which when zero'ed, cause selectedpixel blocks to meet a target compression ratio; and storing P for usein conditioning DCT coefficients.
 6. The method of claim 2 whereinstoring Kframes and their SubFrames after conditioning comprises:Huffman encoding conditioned DCT coefficients for every pixel block of aSubFrame into JPEG codec compatible files; storing meta data relatingeach SubFrame to its respective Kframe; and storing the base line DCTcoefficients of each Kframe.
 7. The method of claim 2 whereinconditioning DCT coefficients for each SubFrame further comprises:configuring a low pass filter (LPF) with an integer F; determining deltafrequency coefficients for each subsequent frame; receiving a minimum Finteger for number of low frequency bins to be passed through at least;receiving a desired compression ratio between a subsequent frame and itskey frame; and applying the configured low pass filter to deltacoefficients of a pixel block, whereby only the coefficient's F lowestbins survive.
 8. The method of claim 7 further comprising: determining afile size for a subsequent frame after Huffman encoding; and, one of onthe condition the desired compression ratio is met, storing thesubsequent frame with meta data pointing to its key frame; and on thecondition the desired compression ratio is not met, selecting the Jframeas a next key frame.
 9. The method of claim 7 further comprising:determining delta coefficients for F low frequency bins; when deltacoefficients for F low frequency bins are insignificant, incrementing Funtil non-trivial delta coefficients are within scope; and configuringthe low pass filter to the post incremented F, whereby the configurablefilter passes a minimum dynamic range of non-zero significant bits. 10.The method of claim 7 further comprising: resetting configurable LPFupon each Kframe selection, whereby the F integer of low frequency binsto be passed through is increased up to the limit of desired compressionratio; and storing the increased F integer until a new Kframe isselected.
 11. The method of claim 7 further comprising: selecting pixelblocks for conditioning; applying the configurable low pass filter toselected pixel blocks; and storing conditioned DCT coefficients as aJframe after Huffman encoding, wherein selecting pixel blocks istriggered by at least one of the group: motion, preselection of views,facial recognition, and external sensors.
 12. A method performed by aprocessor to transform image files by adaptively compressing a series ofsecurity camera frames for transmission through network apparatuses andstorage into non-transitory media, comprising: receiving a sequentialplurality of Joint Photographic Experts Group standard (JPEG) encodedimage files (Jframes); determining discrete cosine transform (DCT)coefficients for every Jframe; selecting at least one key frame (Kframe)from among the Jframes; transforming unselected Jframes into subsequentframes (SubFrames) with respect to an antecedent Kframe; conditioningDCT coefficients for each SubFrame; storing Kframes and their SubFramesafter conditioning; and conditioning by LSB masking of subsequentframes, wherein LSB masking comprises: reverse Huffman coding of aplurality of JPEG codec compatible image files; subtracting each DCTfrequency coefficient from its corresponding key frame equivalent todetermine a delta frequency coefficient; and overwriting a number ofleast significant bits of delta frequency coefficients with anefficiently compressible bit pattern, whereby file compression isachieved by eliminating imperceptible and unnecessary precision.
 13. Themethod of claim 12 wherein LSB masking is applied to C least significantbits of delta coefficients to meet a desired compression ratio between asubsequent frame and its key frame comprising: receiving a desiredcompression ratio; determining delta frequency coefficients for a firstsubsequent frame following a key frame; iterating a number C to mask Cleast significant bits of delta frequency coefficients up to attainingthe desired compression ratio; storing resultant number C until a newkey frame is selected; and masking C least significant bits of eachsubsequent frame until a new key frame is selected.
 14. The method ofclaim 12 wherein LSB masking is applied to A least significant bits ofdelta coefficients to optimize performance of a post-compressionapplication wherein A is selected from among values appropriate toattain a desired success rate for said application which requires arange of precision.
 15. The method of claim 12 wherein LSB masking isapplied to N least significant bits of delta coefficients of selectedfrequency bins on a condition of image noise triggered by rapid randomand non-linear variations in high frequency bins while DC and lowfrequency bins are stable wherein N is selected from among valuesappropriate for device sensitivity to lighting, time of day, andweather.
 16. The method of claim 12 wherein LSB masking is applied to Mleast significant bits of high frequency delta coefficients of a pixelblock on a condition of motion detection triggered by DC and lowfrequency delta coefficients of said pixel block.